AD  A030653 


PANELS  AND  TIME  SERIES  ANALYSIS: 
MARKOV  CHAINS  AND  AUTOREGRESSIVE  PROCESSES 


BY 

( 


T.  W.  ANDERSON 


TECHNICAL  REPORT  NO.  24 
JULY  1976 

PREPARED  UNDER  CONTRACT  N00014-75-C-0442  ' 
(NR-042-034) 

OFFICE  OF  NAVAL  RESEARCH 
THEODORE  W.  ANDERSON,  PROJECT  DIRECTOR 


i 


t 


r~ 

t 

i 


DEPARTMENT  OF  STATISTICS 
STANFORD  UNIVERSITY 
STANFORD,  CALIFORNIA 


J 


!**;*?'**«! 


PANELS  AND  TIME  SERIES  ANALYSIS: 
MARKOV  CHAINS  AND  AUTOREGRESSIVE  PROCESSES 


T.  V.1.  Ar.derson 


Technical  Report  No.  2b 
July  1976 


PREPARED  UNDER  CONTRACT  NOOOU-75 -C-0kk2 
(NR‘-0U2-03U) 

OFFICE  OF  NAVAL  RESEARCH 

Theodore  W.  Anderson,  Project  Director 

Reproduction  in  Whole  or  in  Part  is  Permitted  for 
any  Purpose  of  the  United  States  Government 

Approved  for  public  release;  distribution  unlimited 


DEPARTMENT  OF  STATISTICS 
STANFORD  UNIVERSITY 
STANFORD,  CALIFORNIA 


l 

i 


Panels  and  Time  Series  Analysis: 
Markov  Chains  and  Autoregressive  Processes 

toy 

T.  W.  Anderson 
Stanford  University 


1.  Introduction 

In  a panel  survey  a sample  of  individuals  is  interviewed  at  sev- 
eral points  in  time;  the  resulting  data  are  a sequence  of  responses. 

The  techniques  and  objectives  were  described  by  Lazarsfeld,  Berelson, 
and  Gaudet  in  The  People's  Choice  (19UU),  That  study  was  based  on  re- 
peated interviews  of  many  voters  in  Erie  County,  Ohio,  in  19*+0.  Re- 
spondents were  asked  in  May,  June,  July,  August,  September,  and  October 
for  which  party  (or  candidate)  the  respondent  intended  to  vote.  For 
some  purposes  the  responses  to  this  question  were  coded  as  Republican, 
Democrat,  and  "Don't  Know";  that  is,  each  person  at  each  time  was  put 
into  one  of  three  categories.  The  records  of  the  ^45  persons  who  re- 
sponded to  all  six  interviews  can  be  considered  as  observations 
(or  "realizations")  from  a probability  distribution  of  such  records 
(that  is,  a segment  of  a stochastic  process). 
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A discrete-state,  discrete-time  Markov  chain  can  serve  as  a 
model  for  panel  data.  The  development  of  this  model,  illustrated  by 
the  survey  of  vote  intention,  was  reported  by  Anderson  (195M.  The 
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statistical  methodology,  developed  further  in  collaboration  with  Leo 
Goodman,  appeared  later  in  Anderson  and  Goodman  (1957). 

In  some  panel  surveys  the  responses  may  be  quantitative,  such  as 
answers  to  the  question  how  many  hours  did  you  spend  last  month  reading 
the  newspapers.  In  economic  surveys  the  questions  are  more  likely  to 
produce  numerical  answers:  how  many  hours  did  you  work  last  week  and 

how  much  money  did  you  spend  on  groceries  last  month.  Analysis  of  such 
data  are  sometimes  called  cross-section  studies  by  econometricians. 

A possible  model  for  time  series  consisting  of  measurements  on 
one  or  more  continuous  variables  is  a univariate  or  multivariate  auto- 
regressive process.  The  statistical  methods  for  autoregressive  pro- 
cesses have  been  developed  mainly  for  one  observed  time  series,  that 
is,  the  record  of  one  unit  of  observation.  However,  a characteristic 
feature  of  panel  data  is  that  there  are  available  the  records  of  sev- 
eral units  of  observation.  Of  course,  such  repeated  measurements 
occur  in  other  situations.  A psychologist  may  obtain  a test  score  on 
several  individuals  at  several  points  in  time;  a physician  may  read 
blood  pressures  of  several  patients  daily. 

The  purpose  of  the  present  paper  is  to  review  some  statistical 
methods  for  Markov  chains  and  present  similar  methods  for  correspond- 
ing problems  in  autoregressive  processes  in  the  case  of  repeated 
measurements.  The  statistical  problems  treated  are  those  presented 
by  Anderson  and  Goodman  (1957)  and  suggest  that  other  procedures  for 
Markov  chains  have  their  analogs  for  autoregressive  processes.  The 
development  of  the  methods  for  autoregressive  processes  and  proofs 
of  the  mathematical  statements  will  be  given  in  a later  paper. 
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Statistical  methods  for  a single  observed  series  from  a Markov 
chain*  have  been  discussed  extensively  by  Billingsley  (1961).  The 
autoregressive  process  with  one  observation  on  the  series  is  treated 
in  depth  in  Anderson  (1971). 

Each  section  of  this  paper  is  divided  into  three  subsections, 
the  first  dealing  with  the  Markov  chain  model,  the  second  treating 
the  autoregressive  model,  and  the  third  displaying  correspondences 
between  the  two  models.  Section  2 defines  the  models  and  reviews 
some  of  their  properties.  Section  3 discusses  summary  data  and  es- 
timates of  parameters.  Section  4 develops  tests  of  hypotheses. 

2.  Probability  Models  for  Time  Series. 

2.1.  A Markov  Chain  Model  for  Discrete  Data.  A Markov  chain  can  serve 
as  a model  for  the  probabilities  of  a sequence  of  statistical  variables 
that  take  on  a finite  set  of  values.  Let  the  values  or  states  or  cate- 
gories be  labelled  1,  . ..,  m,  and  let  x^  be  the  statistical  varia- 
ble at  time  t ,t=l,  ...,T  . For  instance,  x^  = 2 might  denote 
an  individual  holding  opinion  2 (Democrat)  at  the  first  interview 
(May).  Then  a Markov  chain  model  specifies  the  probability  of  state 
J at  time  t given  state  i at  time  t - 1 

(2.1)  Pr^xt  = ^xt-l  = = Pijfr)  * i » j = 1,  ....  m . 


| 


* Bartlett  (1951)  developed  some  methods  in  the  context  of  a single 
observation. 
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These  transition  probabilities  satisfy  the  conditions  p. (t)  >.  0 and 

^ J 


(2.2) 


I P^U)  = 1 , 
.1=1  J 


i = 1 , ...,  e . 


While  the  period  of  observation  is  usually  finite  (t=l,...,T)  , 

the  stochastic  process  may  be  defined  for  all  integers, 

t = . . . , -1 , 0 , 1 , ..  . In  any  case,  there  is  a marginal  distribution 

of  the  statistical  variable  at  each  time  point;  the  probability  of 

state  i at  time  t is  denoted  p,.  (t)  [p^(t)  J>  0 , p^(t)  = l]  . 

The  Joint  probability  that  x , = i and  x = j is  p. (t-l)p. , (t ) ; 

X ~ J.  X 1 1 J 

thus,  the  marginal  probability  that  x = j follows  from  the  marginal 


iistribution  at  t - 1 by 


l P, (t-1)  p . (t)  = p (t)  , 
i=l  J J 


J = 1 m 


In  order  to  describe  the  probabilities  of  the  observed  random  variables 

(’  - 1,  ...,  T)  , it  is  only  necessary  to  prescribe  the  vector 

T ' • / = [ p, f 1 ) , p ( 1 ) ] and  the  matrices  P(t)=  [p..(t)],  t=2,...,T 

x ■ ^ x j 

ft.-  x i.stinguishing  feature  of  a Markov  chain  is  that  the  conditional 


pr -hahiJ i oy  of  x(  given  the  entire  past  xf  , x 0,  ...  depends  only 


h.  i : mm<  d iatel  v 


‘ding  variable  x 


the  transition  probabilities  are  homogeneous ; 


► it)  = b 


- • • , -1  , 0,  1 "lien 


; 5 the  marginal  di  stributi ris  are  homogeneous  , that 
process  is  stationary.  In  this  case  (e.l)  is 


•r  atr  ix  : ( tv  ->  [’ 


l r.  i--.  , = f,  , 

i=;  J 


j = 1 , • ■ • , m . 
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shows  that  p is  a left-sided  characteristic  vector  of  P correspond- 
ing to  a characteristic  root  of  1,  that  is,  a root  of 

(2.6)  |P  - XI | = 0 

of  1.  The  equations  (2.2)  can  he  written  Pc  = e , where  £=  (l,...,l),,> 
thus  e is  a right-sided  characteristic  vector  of  P corresponding 
to  a characteristic  root  of  1.  There  are  m roots  X^  = 1,  X^»  • • • » X^ 
of  (2.6);  each  root  satisfies  |X^|  <_  1 . The  Markov  chain  is  called 

irreducible  if  the  root  of  1 is  of  multiplicity  1;  then  there  is  a 
positive  probability  of  going  from  one  state  to  another  in  same  in- 
terval of  time.  In  that  case  p is  determined  uniquely  by  (2.5)  and 
the  normalization  p'c  = 1 . 

In  a more  general  model  the  probability  of  a state  at  time  t 
may  depend  on  the  states  at  several  time  points  earlier.  For  example, 
a second-order  stationary  chain  is  defined  by  the  transition  probabi- 
lities 

( 2 . T ) Pr  {xt  — k ] x^_g  — i , 

where  piJk  >.0  and  E£=1  p^  = 1 , i ,J  = 1 , . ..,  m . Higher-order 
chains  and  nonstationary  chains  are  defined  similarly.  For  some  pur- 
poses it  is  convenient  to  redefine  states  so  as  to  construct  a Markov 
(first-order)  chain  that  is  mathematically  equivalent  to  this  second- 
order  chain.  For  example,  if  m = 2 , let 

(2.8)  xt  * 1 if  xt_1  = 1,  xt  = 1 , 

2 if  xt-1  = 1,  xt  = 2 , 

= 3 if  xt_i  a 2’  xt  = 1 ’ 

14  if  xt-l  2’  xt  = 2 ' 
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Then  the  matrix  of  transition  probabilities  for  ic(t)  is 


(2.9) 


Plll  p112  0 0 

o o p121  p122 

p2u  p212  0 0 

0 0 P221  P222_ 


The  Markov  chain  model  also  includes  multivariate  cases, 
illustration,  consider  two  dichotomous  variables  y^_  = 1 or 
zt  = 1 or  2.  Define  xt  by 


(2.10) 


= i if  yt 

=2  if  y. 


= i,z  = 


, 2. 


= 3 if  y, 

= U if  yt 


As  an 
2 and 


The  model  may  be  further  developed  » , . . i>  effects  of  other 

variables  by  stratification.  If  there  is  a dir •■ret  < conditioning 
variable,  the  transition  probabilities  can  depend  on  it;  that  is, 
the  (homogeneous)  transition  probabilities  in  the  h-th  stratum  may 
be  the  set  [p^J  . 

Prom  a statistician's  viewpoint  the  Markov  chain  model  is  con- 
structed from  a family  of  elementary  multinomial  distributions;  in 
the  case  of  a dichotomous  item  (that  is,  two  states)  these  are  Ber- 


#*?■' I.'**?***'* 


noulli  distributions.  Each  conditional  distribution  is  such  a 


*'■'  ■ 


discrete  distribution.  The  appropriate  statistical  procedures  for 
a Markov  chain  are  similarly  developments  of  methods  for  multinomial 
distributions . 


2.2.  An  Autoregressive  Process.  An  autoregressive  process  can  serve 
as  a model  for  a sequence  of  continuous  random  variables.  In  the  sim- 
plest case  the  (univariate)  stochastic  process  {y^}  has  the  property 
that  the  conditional  distribution  of  y given  y.  , y ,, , . . . , has 
expected  value  B(t)y^  ^ and  y^  - B(t)y^_1  is  statistically 
independent  of  y^  y^  ^ This  is  often  written 

(2.11)  yt  = 8(t)yt_1  + ut  , 

where  {u^}  is  a sequence  of  unobservable  independent  random  vari- 
ables with  means  0.  If  the  autoregression  coefficients  are  homoge- 
nous, that  is  8(t)  = 8 for  some  8 , |8|  < 1 , and  the  u^  are 
identically  distributed,  the  process  is  stationary  and  we  can  write 


(2.12) 


- I esu+_c 

s=0 


2 2 2 
If  the  u^'s  have  variance  a , the  y^'s  have  variance  o /(l-8‘) 


and  covariances 


(2.13) 


* 6,t-S,a2 

5ytys  — 

' s 1-8^ 


If  the  u^'s  are  normally  distributed,  any  set  of  y^’s  are  normal 
and  the  covariance  structure  (2.13)  determines  the  process. 

If  y^_  is  a p-component  vector,  a Markov  (first-order)  vector 
process  is  defined  by 


(2.1U) 


It  = ?(t)^t-l  + "t  * 
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where  B(t)  is  a pxp  matrix  and  (u  } is  a sequence  of  independent 
(unobservable)  random  vectors  with  expected  values  <$u^.  = 0 , covari- 
ance matrices  ^u^u^  = E^  , and  u independent  of  yt_^>  ^t-2’  •••  * 
Let  the  covariance  matrix  of  y be  ^ y y = F . Then  from  (2. it) 
and  the  independence  of  y and  u we  deduce 


(2.15) 


Et  ’ 5(t>  Et-i  ?(t)'  + k • 


If  the  observations  are  made  for  t = 1,  T,  the  model  may  be 

specified  by  the  marginal  distribution  of  y1  and  the  distributions 
of  Ug,  u,j,  . In  particular,  if  y^  and  the  u^'s  are  normal, 

the  model  for  the  observation  period  is  specified  by 
F±,  B(2),  ...,  B(T),  E2,  ...,  Et  . 

When  the  autoregression  matrices  are  homogeneous,  that  is, 

B(t)  = B , and  the  u^'s  are  identically  distributed  with  mean  0 
and  covariance  matrice  E , the  process  is  stationary  if  the  charac- 
teristic roots  of  B are  less  than  1 in  absolute  value  and  the  pro- 
cess is  defined  for  t = . . . , -1 , 0 , 1 , . . . or  y is  assigned  the 
stationary  marginal  distribution.  In  this  case  the  covariance  matrix 
of  y is 


(2.16) 


F = l BSE  b'S  , 
s=0 


and  the  covariance  of  y^  and  y is 

It  "s 


(2.17) 


£yt?s = » 


s < t . 


(Note  that  F = F = F satisfies  (2.15)  for  B(t)  = B and  E = E .) 
A second-order  stationary  autoregressive  vector  process  may  be 


defined  by 


8 


(2.18) 


W 


F 

I: 

’ 

; 


r 


= -1  ?t-l  + ?2  ~t-2  + 'it 


This  model  can  be  written  as  a first-order  process  by  writing 


(2.19) 

It  = 5 

it. 

-1  + "t  ’ 

where 

lit  ' 

/ uA 

(2.20) 

It  - 

I 9 

\0  1 

Ut-lJ 

1 

rE 

B \ 

(2.21) 

B = j 

-1 

\ 

(J 

2 / 

The  characteristic  roots  of  B are  the  roots  of 

(2.22)  |-X2I  + XB1  + B2!  = 0 . 

For  a stationary  process  these  roots  are  to  be  less  than  1 in  ab- 
solute value. 

The  autoregressive  processes  appropriate  to  several  subpopula- 
tions (strata)  may  be  different.  In  the  homogeneous  first-order  case 
the  matrices  of  coefficients  and  the  covariance  matrices  may  be  dif- 
ferent. If  other  influencing  variables  are  continuous,  they  msy  be 
taken  account  of  by  adding  them  to  the  regression  to  yield  the  model. 


(2-23)  ?t  = ? yt-i  + i h + *t  > 

where  z^  is  a vector  of  such  variables  and  y is  a vector  of  para- 
meters. In  particular,  when  - 1 and  y is  a scalar , the  process 
fy^}  may  have  a mean  different  from  0. 


] 
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The  autoregressive  process  is  constructed  from  multivariate  re- 
gressions. In  (2.1^),  for  example,  the  vector  y ^ constitutes  the 
"independent  variables"  and  the  vector  y constitutes  the  "dependent 
variables"  in  ordinary  regression.  To  a large  extent  the  statistical 
methods  for  autoregressive  models  are  regression  or  least  squares 


:.r  re  :ures . 


■ : ■ -•.rrespondence  between  Markov  Chains  and  Autoregressive  Processes. 
■'h<-  :iscre'.e  variable  x which  takes  the  values  1,  ...,  m can  be 
ret  -ace:  by  the  m- component  vector  y in  which  the  i-th  component 

Is  ; If  x * i and  is  0 if  x ^ i . If  we  define  ET  to  be  the 
r.-cor.poner.t  vector  with  1 as  the  i-th  component  and  0 as  the  other 
components,  we  can  define  y as  e when  x = i . Then 


(2.2U) 


Prkt  = S.lkt-i  = ^i}  = Pi.i(t) 


i * j = 1. 


Thus  the  conditional  expectation  of  y given  the  past  is 

(2-2?)  5(ytkt-i*  Xt-2’  •••)  = ftztkt-i'* = ?,(t)Xt-i  • 

i ^ 

If  we  1 = y^  - P(t)  y+_j  , then  (2.25)  implies  Q = 0 and 

(2-26)  ^t  it-s  = ^tAt-l’  ?t-2-“)?t-s  = 2 . 3 = 1.  2. 


Tlie  latter  is  eaui valent  to 


( p . ? 7 ’ 


<Jyt  y 


it  dt.-] 


Note  that  here  has  a singular  distribution  since  £ u^  = 0 . 

Conditional  on  = t:  , the  variance  of  the  j-th  component  of 

yt  is  j ( t ) [ 1-p. j ( t ) ] and  the  covariance  between  the  j-th  and  k-th 
components  is  -p^  j ( t )p^(t ) . The  unconditional  variance  is 
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,Jtt 


£■■  i 

f j 

t 

j ' 

I 


Ei 

ii 


. p . (t-l ) [p . , (t ) i-p.  ,(t )]  , ar,.j  the  unconditional  covariance  is 
i=l  l ij  ij 

~ ^=1  Pi  (t-D  Pij(t)  Pik(t)  . 

The  properties  of  a Markov  chain  are  similar  t . those  of  an 
autoregressive  model,  defined  by  (r.lh),  in  that  the  conditional  ex- 
pectation of  the  observed  vector  y given  the  past  is  a linear 
function  of  the  immediately  preceding  observed  vector  y_  . The 

autc covariances  of  vo  can  be  obtained  from  (?..:'()  ir,  a manner 

~t 

analogous  to  that  of  the  autoregressive  model.  In  particular,  in 

V ' 

the  stationary  case  • , .Li)  holds  with  B replaced  by  F ana  F 


replace 


- ty  4 


t-l  it- 


d^t-l  = 4'£t-]  £t,-l  " l S'  ’ then 


an  • f ..  = - IT 

i k 

representation  of  the  Markov  chair-  differs  from  the  autoregressive 


i'  = p,  - , p.  p.  , and  l ,.  = - 1.7  , p.  p.  , p.,  , ,1  / k . The 

Ji  ,j  i=l  ‘i  ij  c1k  i=l  h i ,1  lk 


process  in  that  the  covariance  matrix  of  the  residual  v is  singu- 
lar an- 1 depends  on  y T in  the  c nditi'na.l  distribution;  while  u 
is  uncorrelated  with  y it  is  not  statistically  independent  of 

y ■ We  also  note  that  the  characteristic  r -ots  f i-  are  less 
than  1 in  absolute  value,  but  one  root  of  i (when  1 is  irre- 
ducible) is  exactly  corresponding  tc  'haruot  eri  Stic  vectors  f 
the  left  and  c n th--  right. ; g is  a gharaeteri  Stic  vector  ..  f 
F corresponding  to  a root  -,f  ■).  in  tfe  fell,  wing  pares  the  para;  '.els 


Estimation  of  Parameters. 


P 

r 


> 


i. 


3. 

3.1.  Estimation  of  Transition  Probabilities.  An  observation  on  an 
individual  consists  of  the  sequence  of  states  for  T successive  time 
points.  For  example,  the  T = 6 successive  monthly  party  preferences 
of  a voter  might  constitute  such  a sequence.  Let  x be  the  state 
of  the  a-th  individual  at  the  t-th  time,  a=l,  ...,Ii,  t=l,  ...,T  . 
The  observed  sequence  x^  , ...»  is  considered  an  observation 

from  the  Markov  chain  specified  by  the  set  of  probabilities 

[p^j(2)],  ...,  [pij(T)]  . The  probability  of  a given  se- 
quence of  states  x ( 1 ) , ...,  x(T)  is 

(3,1)  Px(l)  Px(l),x(2)  (2)  •••  Px(T-l )x(T)  (T^  * 

The  parameters  to  be  estimated  from  a sample  are  the  marginal  prob- 
abilities [p  (l)]  and  the  transition  probabilities 
[p. ,(2)],  ...,  [ p . (T ) ] . The  observed  sequences  are  considered  as 

J J-  J 

independent  observations  from  the  model  defined  by  (3.1). 

Let  n.  (t)  be  the  number  of  observed  individuals  in  state  i 
^ J 

at  time  t - 1 and  j at  time  t , and  let 

m m 

(3*2)  n.  (t-1 ) = I n.  , (t ) = [ n (t-1 ) , i=l,...,m,  t = 2,...,T. 

J=1  J k=l 

The  set  rn.(t),  i,.)  = l,  ...,  m,  for  each  t constitutes  the  fre- 

quencies of  individuals  in  state  i at  time  t-1  and  state  j at 
time  t and  would  usually  be  recorded  in  a two-way  table;  the  row 
totals  are  n^(t-l),  i = l,  ...,  m,  and  the  column  totals  are 

n , ( t ) , J = 1 , . . . , m . 

d 


i 

■ \ 
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A sufficient  set  of  statistics  for  the  node!  (3.1)  is 
n.  (t),  i ,j  = 1 , . . . , m,  t=2,  T ; statistical  inference  need 

only  use  this  information.  The  maximum  likelihood  estimates  of  the 


parameters  are 


(3.3) 


n . ( 1 ) 

--TT-- 


n . . ( t ) 

(3.U)  P..(t)  = ST-  , 

lj  n^ ( t-1 1 


i , J = 1 » • • • > ®i  t = 2 , . . . , T 


[if  n^(t-l)  = 0,  then  n^.(t)  = 0 and  (3.*0  is  undefined.]  The 
estimates  are  in  effect  estimates  of  multinomial  probabilities.  The 
estimates  satisfy  p.  ( 1 ) >.  0 , E?  , p.  (l ) = 1,  p . . (t ) > 0 , and 

E"«i  V11  ■ 1 • 

Now  consider  the  case  of  homogeneous  transition  probabilities, 
but  with  the  initial  probabilities  p^l)  arbitrary.  Then  a suffi- 
cient set  of  statistics  is  n^l),  i = 1,  ...,  m,  and 


(3.5) 


l n (i)  , i,j  = 1,  ... , m . 

t=2  " 


The  two-way  table  of  frequencies  (3*5)  Is  the  sum  of  the  T-1 

two-way  tables  with  entries  n . (t)  . [However,  n.(l)  is  not  a 

lj  i 

marginal  total  of  this  table.]  The  maximum  likelihood  estimates  are 
(3.3)  and 


(3.6) 


n* 

1 


where  n*  = E*?  , n.  . . 

i ij 


In  an  alternative  model  the  states  at  the  initial,  time  t = I 


are  considered  as  given;  that  is,  x , a = i,  . ..,  N , are  treated 
as  nonstochastic.  Then  n^(l),  i = 1,  . ..,  m , are  considered  as 


given,  that  is,  are  parameters,  not  statistics.  The  sufficient  uet 

of  statistics  is  the  set  rojCt),  ^ = ^ , . . . , T , or 

r..,,  i,j  n as  the  case  may  be. 

In  the  case  of  homogeneous  transition  probabilities,  it  may  be 

desired  to  treat  the  process  as  stationary;  the  marginal  probabi • ity 

distribution  [p.]  is  determined  by  ip..]  as  the  solution  fo  JO; 
r i J 

in  particular,  the  initial  distribution  EpOl)}  must  be  [p.]  • 

A sufficient  set  of  statistics  is  n^d),  i =1,  . ..,  m , ana 
, i,j  = ],  ...,  m . While  the  parameter  set  car  be  reduced  to 
[p  ] , the  maximum  likelihood  estimates  are  not  (3.6)  in  this  case 
because  the  likelihood  function  depends  on  (l ) and 
P A 1 ) = P|  > i = 1»  ...,  m,  the  latter  being  functions  of  [p^’J  . 

The  estimates  are  too  complicated  to  give  explicitly. 

To  assess  sampling  variability  and  to  evaluate  confidence  is 
inferences  it  is  desirable  to  know  the  distributions  of  the  estimates. 
Since  the  exact  distributions  for  given  sample  sizes  arc-  too  com- 
plicated to  be  useful  we  consider  "large-sample  theory".  Anderson 

and  Goodman  (1957)  developed  asymptotic  theory  for  the  number  of 
observations  N getting  large;  this  large-sample  the-  ry  is  appr-  r riate 
fo-  panel  studies  where  the  number  of  time  points  is  sma i 1 (soi.iel.it.es 
T = 2)  and  the  number  of  respondents  is  large.  When  the  transit:  r. 
probabilities  are  homogeneous , the  parameters  do  not  depend  or.  the 
time  t except  the  initial  probabilities  [p.(.l)]  ; then  it  is 

meaningful  to  consider  asymptotic  distributions  as  T * ■ . This 


theory  is  appropriate  when  the  data  consist  f one  ( or  several)  : ng 


It 


series;  the  measurements  are  not  necessarily  repeated.  Bartlett  (1951) 
gave  some  of  this  theory.  In  general,  when  a limiting  distribution 
hold  for  T -*  00  it  will  hold  for  arbitrary  N ; in  fact,  with 
proper  normalization  the  same  limiting  distribution  holds  for  N -*■  00 
and  fixed  T . In  such  a case  we  will  say  the  limit  holds  as  N 
and/or  T -*•  00  . [In  mathematical  terms  the  error  is  arbitrarily 
small  if  N is  sufficiently  large  or  T is  sufficiently  large,  or 
both. ] 

The  asymptotic  theory  for  N -*•  00  is  the  usual  multinomial 

theory,  for  the  N observations  on  the  chain  are  independent.  Then 

by  the  usual  multinomial  theory  ✓'N'fpr  (l  )-p^  (l ) ] , i =1,  ...,  m , 

have  a limiting  normal  distribution  with  means  0 and  covariance 

matrix  [p.(l)  S.  -p.(l)  p ( 1 ) ] , where  6 . . = 1 and  6 . = 0 , i # J . 
i ij  1 j ti  ij 

(t)-p  (t)]  , j = 1 m , 

have  a limiting  normal  distribution  with  means  0 and  covariance 

matrix  [p . (t)6  , p . (t )p. , (t ) ] ; the  sets  for  different  values 
ij  jk  * ij  ik 

of  i and/or  different  values  of  t are  independent  in  the  limiting 
distribution.  The  limiting  distribution  of  the  estimates  of  the 
rows  of  the  transition  probability  matrices  is  the  same  as  that  of 
estimates  of  independent  multinomial  distributions. 


If  p^Ct-l)  > 0 , the  set 


If  the  transition  probabilities  are  homogeneous  and  the  chain 

is  irreducible,  then  for  each  i the  set  brT'.p.  -p.  , ) , i = 1 , . . . , m , 

1 1 J 1 J 

has  a limiting  normal  distribution  with  means  0,  variances  p^Q-p^) 

and  covariances  -p, ,p.,  , j # k , and  the  sets  for  different  values 
rij  ik  0 

of  i are  independent  in  the  limiting  distribution.  The  limits  in 
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the  homogeneous  case  are  valid  as  tl 


-►  Of; 


and / or  T ~ 


3.2.  Estimation  of  Autoregressive  Coefficients.  I.eV  y+  he  the 
p-component  vector  o;'  measurements  of  the  a-th  individual  at  tr.e  t-th 
time  point,  a = 1,  N,  t = ],  T . The  model  is  a first- 

order  autoregressive  model  (2. lit)  with  u having  the  normal  dis- 
tribution N ( 0 , 1 ) and  y having  the  norma:  distribution  l.'(0,ir,  ) . 

The  probability  density  of  the  sequence  y , . . . , y for  a given 

~ lOt  — L Oi 


(3.7) 


irpp  i.  T — 

(2t:)2  |f  | 2 n |z  |2 

~ t=2 


x expJ-^-y'  F"V  + \ (y  -B(t)y  ) r”1(.v  - P(t)y  , ) 

I 2[~la~l  Hot  t^2  ~ta  ~ it-1 ,a  ~t  -t  - 


Then  a sufficient  set  of  statistics  for  F 


ltb(i b(t),?:2 


• N i ’ 

is  Y.  , y,  y,  , t = 1,  . . . , T , and  n , y , y , t = . . 
a=l  ~ta  -tor  a=x  ~t-i  ,a  -iu’ 

The  maximum  likelihood  estimates  of  the  oarameter  matrices  are 


(3.8) 


f = — y v y 1 

~1  N L liu  11  a ’ 

a=l 


0.9)  Bit)  :ytI?u,J 

a=l  \a=l 


(3.10)  r 


» • • • » y 


,a  K,,  ) • 


it) 


The  components  of  B(t)  are  least  suuares  estimates.  [See  Anderson 
(1958),  Chapter  8,  and  Anderson  (1971),  Chapter  5,  for  example.] 


The  assumption  that  the  autoregression  matrices  are  homogeneous 
and  the  disturbances  identicaily  distributed  leads  to  considerable- 


simplification.  The  sufficient  set  of  statistics  for  F , B,  and 


E is 


rN 

<L'a='l  ~la  -la 


T "i  , 

Zt-2  ^1  Ua  It, 


and 


VjfI  ita 


The  maximum  likelihood  estimate  of  is  (3.8)  and  the  maximum 


likelihood  estimates  of  the  other  matrices  are 
T N / T ?! 


.-1 


(3.11) 

(3.12) 


t -?.  a=I 

;P  \T 


= M(T-l)  tl2  Jx(?to"  B -t-l  ,a)  ( -i  a"  P ?t-l  ,a) 


N(T-l) 


1 N T N 

T l y y’  - b l y V v*  B 


An  alternative  mo  -.el  is  t ; consider  y.  a = l,  ...,  N , as 


nonstochas+.ic  or  fixed  and  treat  \’  , r.  = p,  ...  t n = 1 rj 

conditional  1 y . Then  tr.e  maximum  likelihood  e.- 1 i mates  of  B and  E 
are  ' i . 1 1 ) and  (3.1.). 


When  y is  considered  to  have  the  marginal  normal  distribution 
determined  bv  the  stationary  process,  the  covariance  matrix  F^  is 
a function  of  b ana  E as  given  oy  (2. l6).  Then  the  maximum  like- 
lihood estimates  are  much  more,  complicated.  (As  T -*•  ~ , (3.1l)  and 
(3.12)  are  asymptotically  equivalent  to  the  maximum  likelihood  esti- 
mates, but  not  as  N -*•  ■»  .) 


We  now  consider  the  asymptotic  properties  of  the  estimates  as 
N -*■  oo  and/or  T -*•  <*>  . As  N ->  » , F^  , B(t)  , and  E are  consis- 
tent estimates  of  F^  , B(t),  and  E^.,  t = 2,  . T , respectively. 
The  elements  of  v^f[B(t)  - B(t)]  have  a limiting  norma]  distribution 
with  means  0 and  covariances  constituting  the  Krcnecker  product 
E^  © F^i  , t = 2,  T . The  matrix  F ^ is  estimated  consis- 


tently by  (i/n)^  yt_lfQ  y;_1>a 


In  the  case  of  homogeneous  autoregressive  coefficients,  regard- 
less of  the  distribution  of  y^,  ot  * 1,  . ..,  N , arid  of  the  value 

A /v 

of  N , as  T -*•  oo  £ and  Z are  consistent  estimates  of  B and 
Z , respectively,  and  /F[B  - B]  has  a limiting  normal  distribution 
with  means  0 and  covariances  constituting  (l/N)  E © F ± . The 
matrix  F is  consistent]  estimated  by^ 

*[l/N(T-l ) ] Et=2  Ea=1  yt_1>a  y^-l.a  • 

If  T is  fixed  and  N -*■  00  , ^(B  - B)  has  a limiting  normal 
distribution  with  means  0 and  covariances  constituting 


(3.13) 


oil  zX1 


3.3.  Correspondence  of  Sufficient  Statistics  and  Estimates  in  Markov 

Chains  and  Autoregressive  Processes.  Tn  Section  2.3  a Markov  chain 

was  represented  as  a vector  process  with  y = £ with  conditional 

~t  ~ j 

probability  p..(t)  given  y.  , = £.  . From  this  definition  we  can 
lj  ~t-l  ~i 

write  the  second-order  moment  matrices  for  the  Markov  chain  as 


(3-1U) 


? y y = [ n.(t)e.  e.  , 

h .to.  ~ t«  . L , i ~ i i 
a=l  i=l 
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1 


which  is  a diagonal  matrix  with  r:,(t)  as  the  i-lh  diagonal  element, 
and 

(3.15) 


N , m 

2,  - . I 3! 

01  = 1 ’ 1 ,j  = l J “ 


which  is  an  mxm  matrix  with  n.,(t)  as  the  i ,j-th  element.  IJote 
that  the  elements  of  (3. 11*)  can  be  derived  from  the  elements  of  (3.15) 
Here  (3.15)  for  t = , . ..,  T constitute  a sufficient  set  of  sta- 

tistics for  the  nonhornogeneous  Markov  chain.  The  estimates  (3. it) 
constitute  elements  of  the  matrix  estimates  (3.9)  under  this  corre- 
spondence. 

If  the  transition  probabilities  are  homogeneous  with  arbitrary 
initial  probabilities,  a sufficient  set  of  statistics  is  (3.110  for 
t = 1 and 

, „ T N m 

3'16'  ^ I ?t_i  a yt a = l nii  • 

t=?  ot=l  t i,a  01  i,j=i  ^ '•  '•J 
The  estimates  (3.6)  are  elements  of  (3.1l). 


O Tests  of  Hypotheses. 

U.l.  Tests  for  Markov  Chains.  The  tests  for  Markov  chains  presented 
in  this  section  were  developed  by  Anderson  and  doc dm an  (1957);  they 
can  be  applied  for  any  value  of  T (>.  H)  and  for  large  N . Bart- 
lett (l95l)  developed  some  of  the  tests  as  valid  for  one  observed 
sequence  of  states  from  a homogeneous  chain  when  T . For  con- 

venience, we  shall  ass'ime  p..(t)  > 0 and  p.  . > 0 as  the  case 

ij 

may  be.  Some  of  the  procedures  were  illustrated  in  Anderson  (195M- 
Jn  Section  U.2  test  criteria  for  tlie  corresponding  hypotheses  for 
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autoregressive  models  are  given  (in  the  same  sequence)  and  in  Sec- 
tion i*,3  the  correspondences  are  discussed. 

Specified  Probabilities.  In  the  homogeneous  chain  to  test  the 

null  hypothesis  that  p.  = p?  , j = 1 , ,,,,  m,  vhere  the  set  p° 

J-J  ij  J ij 

are  specified,  for  a given  i one  can  use  the  criterion 

m (p . - p°  )2 

(U.l)  n*  l ^LL_ 

i . 


J*1  pT, 
ij 


° O 

which  under  the  null  hypothesis  has  a limiting  x -distribution  with 
m - 1 degrees  of  freedom  as  N -*■  ® and/or  as  T . The  criteria 
for  different  i are  asymptotically  independent.  To  test  the  null 
hypothesis  p^  = , i = •••»  m»  the  sum  of  { U . 1 ) over  i can 

he  used;  it  has  a limiting  x -distribution  with  m(m-l)  degrees  of 
freedom  when  the  null  hypothesis  is  true.  If  the  transition  probabi- 
lities are  not  necessarily  homogeneous  one  can  test  the  null  hypothesis 
= J = 1»  •••>  m,  for  given  i and  t by  use  of  the 


criterion 


(U.2) 


j=i  P°j(t) 


O 

which  under  the  null  hypothesis  has  a limiting  x^-distribution  as 

N -*•  00  • The  criteria  for  different  i and  t are  asymptotically 

independent;  they  can  be  summed  over  i and/or  t to  form  xc- 

criteria  for  combined  hypotheses.  These  criteria  are  analogs  of  the 
2 

X goodness-of-fit  criterion  for  multinomial  distributions.  The 
test  procedures  can  be  inverted  in  the  usual  fashion  to  obtain  con- 
fidence regions  for  the  transition  probabilities. 
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Homogeneity . Ir.  treating  r'-nr-;  a tin.  1 river-. ti gat  'r  may  ques- 
tion whether  conditions  change  n-.’&r  trie  time  of  observation 

to  require  the  use  of  an  inherogene.  a.-  chair..  To  test  tire  null  hy- 


pothesis that  p . . ( t 


p , t = , , for  some  p , 


i , J =1,  m,  one  may  use  the  criterion 


(*.3) 


T n.  [pM(-)-fu,r 

I 1 n.(t-l)  J — T 

t=2  i * Fi,j 


Under  the  null  hypothesis  this  criterion  has  a limiting  x^-distributior. 
with  m(m-l)(T~2)  degrees  of  freedom  as  N -*  00  . If  one  thinks  of 
the  set  of  probabilities  p^.(t)  and  the  set  of  frequencies  n^(t) 
in  (three-way)  mXmx(T-l)  arrays,  the  criterion  (h.'j)  is  the  usual 
X^’-cr  iter  ion  for  testing  the  independence  of  the  categorization 
(i,j)  and  the  classification  t . 

independence . If  p.  . = p , i = 1 , for  some  p.  , 

1 o J J 

.1  = 1,  ....  m,  the  random  variable  x.  is  independent,  of  x,  , 

t * t-1 

in  a homogeneous  Markov  chain.  Tc  test  the  null  hypothesis  of  in- 
dependence one  may  use  the  criterion 


(MO 


vherr- 


.1 


(M) 


- - ni.i 
p,i  _ n(t-i) 


Under  the  null  hypothesis  the  criterion  has  a limiting  x -distribution 

2 

with  (m-l)  degrees  of  freedom  as  N -*■  <*>  and/or  T -*■  03  . The 

2 

criterion  is  the  x -criterion  for  i r.dependenee  in  the  two-way  table 

fau>  ■ 


Given  Order.  An  investigator  may  consider  a more  elaborate 
model  in  which  the  probability  of  a state  observed  at  time  t depends 
on  the  state  observed  in  the  last  r time  points.  He  may  question 
whether  it  would  be  appropriate  to  use  a model  of  lower  order.  In- 
dependence is  order  r = 0 ; in  the  previous  section  this  hypothesis 
was  tested  against  the  alternative  that  r = 1 . As  another  example, 
we  consider  testing  the  null  hypothesis  that  a homogeneous  second- 

order  chain  is  first-order,  that  is,  that  p.  for  some 

i,ik  1 jk 

suitable  Pjk(Pjk  > 0,  £k=1  pJk  = 1)  • 

In  a second-order  homogeneous  chain  with  n.(l)  and  n.,(2) 

1 l j 

as  given  [or  p.(l)  and  p..(2)  as  arbitrary]  the  maximum  like- 
i i J 

lihood  estimates  of  p.  , are 

1 JK 


(4.6) 


pijk  = ^’  i,j,k  = l. 


where  n.Jk  = n. Jk(t)  , n. 


ij  = Zk=l  nijk  ’ and  nijk(t)  iE 


the  number  of  observations  of  state  i at  t - 2,  J at.  t - 1 , 
and  k at  t . Then  a criterion  for  testing  that  an  assumed 


second-order  chain  is  actually  a first-order  chai..  is 


(4.7) 


l < 


* " pjk) 


i » j >k=l 


(4.8) 


„m 

1.  , n. 
i=l  ilk 


Jk  I?  , n* . 

i=J  ij 


When  the  null  hypothesis  is  true,  (4.J)  has  a limiting  x^-distribution 


with  m(m-l)  degrees  of  freedom  as  N -+  00  and/or  T + 00  . 


ii 


Several  Chains  i cent ical . A population  may  be  stratified  into 
several  subpopulations  and  the  transition  probabilities  may  be  dif- 
ferent. Suppose  we  have  samples  from  s Markov  chains  with  tran- 

and  we  wish  to  test  the 


(pi 

sition  probabilities  ^ ' , h = 1 , s. 


null  hypothesis  that  the  chains  are  identical,  that  is,  that 


(h) 


~(h)  . +. 

p.  . be  the  max- 


p . = p . . , h = 1 , . . . , s , f or  s ome  p . . . Le 
J C ij  i,i  i j 

imum  likelihood  estimate  of  the  transition  probability  from  the 

h-th  sample,  and  Jet  pf  . ^ be  the  estimate  based  cn  all  s samples 

under  the  assumption  of  the  null  hypothesis.  The  criterion  is 

s m . [p00  - p!0]2 

(-9)  I I n*(n)  ^ - , 


Pij 


h=l  i , j=l 

which  has  the  x~-distribution  with  (s-l)m(m-.i)  degrees  of  freedom 
under  the  null  hypothesis  as  N * 00  and/or  T =>■■  . 


Independence  of  Two  Sets  of  States.  Suppose  the  state  of  a 
Markov  chain  is  determined  by  a pair  of  responses  (that  is,  answers 
to  two  questions).  Denote  the  state  as  (a, 3),  a = I , . . . , A,  and 
3 = 1,  ...,  9,  where  a.  denotes  the  first  answer  or  class  and  3 
the  second,  and  the  transition  probabilities  as  p „ .Is  the 
sequence  of  changes  in  one  classification  independent  of  that  in  the 
second?  The  null  hypothesis  is 

U-i°)  vx3,pv  = qctp  r3v  ’ a’,J  A’  3 = 1 > ••  • ’ B i 

where  q is  a transition  probability  for  the  first  classification 

and  r.,  is  for  the  second.  Let  n „ (t)  be  the  n’unber  of  indi- 

Bv  aS  ,uv 

vi duals  iii  state  (a,  3)  at  t.  - 1 and  (p,v)  at  t , and  let 


T 

n q = E.  _ n o (t)  . The  maximum  Likelihood  estimate  of  p 0 
a3,yv  t=2  a8,;.v  *n3,fjv 


(b.ll)  p 


' aB,yu 


aB  ,yv  „A  VB 

''Y=l  'J5=l  naB,y6 


, a,y  = L,. . . ,A,  S,Y=  1,.  • . ,B, 


when  the  null  hypothesis  is  not  assumed  and  is  q rv,  , where 

ay  3v 


(U.12)  3 


yB 

3.v=I  na  3 


iii: 


ap  A ..B 

Y-l  "B,v=l  na3  ,YV 


Ja,lJ=I  r‘a3,pv 

Bv  _A  „B  

> . L-  , n r, 

a,p=l  i =1  a3,yi 


when  the  null  hypothesis  is  assumed.  The  x^-critericn  for  testing 
the  null  hypothesis  of  independence  is 


( U . 1 3 ) 


V ? * (gqg.uu  - %xu  V‘ 

l l n0C'3  3 p 

a ,b=l  S,v=l  try  rBv 


where  n*.,  = . £?  , n 0 , t • ''Tien  the  null  hypothesis  is  true, 

aB  Y=1  o=l  aB,Yo 

2 

the  criterion  has  a limiting  x -distribution  with 

AB( AB-1 ) - A(A-1 ) - B(B-1 ) = (A-l)(B-l)  (AB+A+B)  degrees  of  freedom 

as  li  -*  « and/or  T ■+  *>  . 


1.2.  Tests  for  Autoregressive  Processes.  The  development  and  appl i- 
cation  of  procedures  is  based  on  asymptotic  theory  as  N -*•  00  and 'or 
T -*■  m . In  this  section  we  study  met  in  ds  of  testing  hypotheses 
about  the  matrices  of  autoregressive  coefficients  and,  in  some  cares, 
hypotheses  about  covariance  matrices.  The  hypotheses  correspond  to 
those  concerning  Mark  v chains  presented  in  Section  U .3  . The  proce- 
dures are  analogs  of  procedures  in  multivariate  regression.  (bee 
Anderson  '1958),  Chapter  8,  for  example.]  For  many  hypotheses  * hr  -e 


N * 00  when  The  null  hypothesis  is  true. 


Independence . In  the  first--,  rder  autoregressive  model  indepen- 
dence at  uit'l'erent  time  points  is  identica-  to  B(?)  = ...  = B(T)  = 0 
riven  = ...  = F.r  , cr  in  the  homogeneous  case  B = 0 . Ir.  the 
Latter  case  the  null  hypothesis  of  independence  is  that  the  autore- 
res..; i on  matrix  is  the  zero  matrix  and  the  criterion  is  with 

L =0  , that  is, 


t=2  a=l 


?t-l,a  ~t-l ,a  ? 


Given  Order.  The  second-order  homogeneous  process  is  defined  by 
(2.29) • It  is  a first-order  process  if  B0  = 0 . To  test  this  null 
hypothesis  we  need  estimates  of  and 


ter  * 


A criterion  fur 


lag  the  rmJ  1 hypot.hesir 


L.  Llt-2,a 
t=  3 a=l  ’ 


Under  the  nul 1 hype. thesis  th*  criterion  nus  n i imiTiug  \"-dist ritution 
with  p"  degrees  of  freedom  as  and /or  7 * A . 


Several  Processes  Identical.  Consider  testing  the  null  hypothesis 
that  the  matrices  of  autoregressive  ■-oefficier.t  i:  vf  ::  first-order 

homogeneous  autoregressive  processes  wi  t n lie;. '.leal  rnviri  ar.ee  matrices 


I are  equal  on  the  basis  of  N,  obr-cned  time-  r"ries  of  .Length  1 

'(h) 

from  the  h-th  process,  h = 1 , s . If  P is  the  estimate 

of  the  matrix  for  the  h-th  process  , b is  the  pooled  .t  titrate  of  the 
hypothetically  equal  matrices,  and  )'  is  the  poled  • inate  of  Y.  , 


a criterion  for  testing  the  null  hyjy  1 hesis  1.-: 


( i- . 20  ) 
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d '-green  of  freedom. 


(l) ' (?) ' 

Independence  of  Two  Subprocesses.  Suppose  = (y^  y^ 


and  we  ask  the  question  whether  the  first-order  autoregressive  proces 


is  such  that  the  two  subprocesses  {y^}  and  {yf2^}  are  indepen- 


dent . Let 


k.21) 
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p11 
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, l = | 
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The  two  subprocesses  (in  the  Gaussian  case)  are  independent  if  and 
only  if 


(h .22) 

(b.23) 


?12  0 ’ ~ 0 » 
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The  estimates  (3.1l)  and  (3.12)  are  partitioned  similarly.  A test  of 
the  hypothesis  ( U . 23 ) can  be  based  on  the  criterion 


(U.2U) 


H(T-l)  tr  ill  111  ?21 


which  has  a limiting  x -distribution  with  p p2  degrees  of  freedom, 
(as  f!  ■*  °°  and/or  T ->  °°)  , where  p^  is  the  numoer  of  coordinates 

(i)  _ ........ ...  ._0?) 


m y 


+ and  pg  is  the  number  in  y 
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W'.en  1^2  = ^2i  = ? ’ a criterion  to  test  the  null  hypothesis 
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The  three  criteria  are  asymptotical  ly  incieper.der  t . 

■i.3.  Correspondence  Terts.  The  r.H*  rix  P has  m(w-l ) element; 
i.i  aoecify  because  the  sun  of  elements  in  each  row  i ;•  .L , while  1 he 
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as  criteria  for  specified  matrices. 


Since  n.(t  - l)  is  the  i-th  lifu---»naJ  ‘'.cm.-ff  of 
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For  independence  (1.1)  and  ( 1 .1 6 ) correspond ; in  the  discrt  i ■■ 

’■ire  the  covariance  matrix  of  p.  . lots  not.  iletiend  on  l . lie 

i J 

,\. rrespondence  between  degrees  of  freedom,  however,  is  (r.  - 1 )'  and 
t 'recause  independence  in  the  discrete  case  is  not  defined  by 

I 

set  ring  P = 0 . (independence  is  P = £ p . ) 

For  testing  whether  a second-order  model  is  actually  first-  rder 

/■> 

criteria  1.1.7)  and  ( h . 19 ) correspond.  In  the  latter  Y.  could  be 
replaced  by  (3.12),  which  is  a consistent  estimate  of  Y.  when  the 
null  hypothesis  is  true,  or  Y could  be  replaced  by  the  matrix  in 

>•  rackets  divided  by  tl ( T - 2)  . Note  the  degrees  ef  freedom  are 

, ? 2 

" " - IJ  and  p , respectively. 

To  test  equality  of  matrices  criteria  (it. 9)  and  (1.20)  corres- 
’ a i.  Tne  tests  of  independence,  however,  do  not  have  similar  st rue- 
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